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CGM  Validation  Test  Service,  Section  2.3.3  Disputed  and  withdrawn  tests,  page  8,  paragraph 
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ABSTRACT 


The  Computer  Graphics  Metafile  (CGM)  standard  defines  a file  format  for  describing 
2-dimensional  pictures  in  a device  independent,  computer  readable  format.  As  a data 
interchange  standard,  CGM  is  suitable  for  the  storage  and  exchange  of  graphical  data  (i.e., 
pictures)  among  different  computers,  graphical  devices,  and  applications. 

This  document  provides  general  procedures  for  the  National  Institute  of  Standards  and 
Technology’s  (NIST)  CGM  Validation  Test  Service.  The  NIST  CGM  Validation  Test  Service 
provides  a way  of  determining  the  degree  to  which  an  implementation  conforms  to  the  CGM 
standard  (Federal  Information  Processing  Standard  128-1,  Computer  Graphics  Metafile,  and  the 
Continuous  Acquisition  and  Life-Cycle  Support  Application  Profile,  MIL-D-28003A).  The  goal 
of  the  NIST  Test  Service  is  to  maximize  the  probability  of  successful  interchange  of  CGMs 
between  conforming  systems. 

Conformance  testing  is  a functional  and  fiscal  advantage  to  the  vendor  and  end  user.  The  end 
user  will  be  provided  with  increased  product  confidence  and  expanded  freedom  in  the  market 
place.  The  vendor  will  benefit  from  the  test  service  by  providing  a validated  product,  thus 
stimulating  sales  to  both  government  and  industry. 

The  document  is  divided  into  three  testing  programs:  metafile  testing,  generator  testing,  and 
interpreter  testing.  In  order  to  take  into  account  the  differences  among  the  three  testing 
programs,  many  procedures  have  been  tailored  to  the  specific  testing  program.  The  policies  and 
procedures  presented  in  this  document  are  organized  into  five  sections:  an  introduction,  general 
procedures,  and  specific  procedures  for  each  of  the  three  testing  programs. 


Keywords 

Computer  Graphics  Metafile  (CGM),  Continuous  Acquisition  and  Life-Cycle  Support  (CALS), 
conformance  testing,  graphics  standards,  validation,  metafile,  generator,  interpreter 


v 


1 Introduction 


1.1  Purpose 

The  testing  of  Computer  Graphics  Metafiles  (CGMs)  to  determine  the  degree  to  which  they 
conform  to  the  federal  standards  may  be  required  by  government  departments  and  agencies  in 
accordance  with  Federal  Information  Resources  Management  Regulation  (FIRMR)  201-20.303, 
201-20.304, 201-39. 1002,  the  associated  Federal  ADP  and  Telecommunications  Standards  Index, 
and  as  specified  by  FIPS  128-1. 

As  part  of  its  mission,  the  Computer  Systems  Laboratory  (CSL)  provides  a validation  service 
for  Federal  Information  Processing  Standards  (FIPS)  in  support  of  government  departments  and 
agencies  procurement  requirements.  This  document  provides  the  operating  procedures  for 
administering  the  CGM  validation  service. 

The  CGM  validation  service  is  divided  into  three  testing  programs:  metafile  testing,  generator 
testing,  and  interpreter  testing.  In  order  to  take  into  account  the  differences  between  the  three 
testing  programs,  many  procedures  have  been  tailored  to  the  specific  testing  program.  Thus, 
the  procedures  presented  in  this  document  are  organized  into  five  sections: 

— Section  1,  introduction, 

— Section  2,  those  that  apply  to  the  test  service  as  a whole, 

— Section  3,  those  that  apply  to  the  metafile  testing  program, 

— Section  4,  those  that  apply  to  the  generator  testing  program,  and 

— Section  5,  those  that  apply  to  the  interpreter  testing  program. 


1.2  The  CGM  standard 

The  Federal  Information  Processing  Standards  (FIPS)  Publication  128-1,  Computer  Graphics 
Metafile,  adopts  the  American  National  Standards  Institute/Intemational  Organization  for 
Standardization  (ANSI/ISO)  Computer  Graphics  Metafile,  ANSI/ISO  8632. 1-4: 1992.  ANSI/ISO 
8632.1-4:1992  defines  three  versions  of  the  Computer  Graphics  Metafile.  FIPS  128-1  adopts 
all  three  versions. 

Additionally,  the  FIPS  128-1  requires  the  use  of  application  profiles.  In  particular,  the  FIPS 
128-1  requires  the  use  of  military  specification  MIL-D-28003,  commonly  known  as  the  CALS 
(Continuous  Acquisition  and  Life-Cycle  Support)  CGM  Application  Profile  (AP). 
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1.3  Scope  of  validation 

Validation  is  the  process  of  testing  an  implementation  for  conformance  to  a standard.  In 
particular,  current  CGM  validation  is  the  testing  of  the  metafile,  generator,  or  interpreter  for 
conformance.  If  compliance  is  demonstrated,  a Certificate  of  Validation  is  issued. 

The  validation  processes  for  metafiles,  generators,  and  interpreters  are  three  distinct  processes. 
For  metafiles,  the  validation  focuses  on  testing  an  instance  of  a CGM,  that  is,  a CGM  data 
stream  for  compliance  to  Version  1 CGM  as  specified  in  the  FIPS  128-1  and  the  CALS  CGM 
AP.  For  generators,  the  validation  focuses  on  testing  the  CGM  generator  and  its  operating 
environment  for  compliance  to  Version  1 CGM  as  specified  in  the  FIPS  128-1  and  the  CALS 
CGM  AP.  For  interpreters,  the  validation  focuses  on  testing  the  CGM  interpreter  and  its  ability 
to  correctly  and  completely  parse  a CGM  in  compliance  to  Version  1 CGM  as  specified  in  the 
FIPS  128-1  and  the  CALS  CGM  AP. 


1.4  Definitions 

The  following  terms  are  used  throughout  this  document. 

Certificate  of  Validation  — a certificate  that  acknowledges  compliance  of  an  implementation  to 
a FIPS. 

CGM  generator  or  generator  — the  program,  process,  or  product  that  can  write  metafiles,  which 
represent  a graphical  picture  assembled  from  information  gathered  or  calculated  by  the 
application  software  or  calling  program. 

CGM  interpreter  or  interpreter  — the  program,  process,  or  product  that  can  correctly  and 
completely  parse  a CGM  and  produce  the  intended  picture. 

Client  — anyone  requesting  conformance  testing. 

Compliance  — the  state  of  the  implementation  for  which  correct  test  results  were  obtained  using 
the  applicable  version  of  the  test  suite. 

Computer  system  environment  or  operating  environment  — the  combination  of  computer 
hardware,  software,  graphics  applications,  programs,  etc. , used  during  the  testing  process 
to  support  the  generator. 

Conformance  testing  the  tests  to  evaluate  the  adherence  or  non-adherence  of  an 
implementation  under  test  to  a standard. 
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Control  board  — the  body  of  CGM  and/or  validation  experts  who  resolve  disputes  concerning 
the  correctness  of  the  test  suite  with  respect  to  the  CGM  standard  and  application  profile. 
The  control  board  is  composed  of  testing  laboratory  members. 

Implementation  — For  metafile  testing,  the  metafile  is  the  implementation.  For  generator  and 
interpreter  testing,  the  generator  and  interpreter  are  the  implementation. 

Implementation  under  test  (lUT)  — For  metafile  testing,  the  implementation  under  test  is  the 
metafile  being  tested  for  conformance.  For  generator  or  interpreter  testing,  the 
implementation  under  test  is  the  combination  of  CGM  generator  or  interpreter,  and  the 
computer  system  environment  required  to  support  them. 

Registered  Report  — the  VSR,  once  all  validation  processing  steps  have  been  completed.  As  a 
Registered  Report,  the  VSR  is  entered  on  the  Validated  Products  List. 

Test  laboratory  — the  laboratory  performing  the  validation,  in  this  case,  NIST. 

Validated  Products  List  (VPL)  — the  list  published  by  NIST  of  computer  products  that  have 
been  validated  for  conformance  to  specific  FIPS. 

Validation  — the  process  of  testing  for  conformance. 

Validation  Summary  Report  (VSR)  — the  document  which  presents  the  test  results  and  other 
information  relevant  to  the  tests. 
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2 General  procedures 


2.1  Validation  by  testing 

2.1.1  Overview 

The  process  of  validation  by  testing  consists  of  a test  laboratory  conducting  the  conformance 
tests  on  a client’s  implementation  and  reporting  the  results  of  that  testing  in  a Validation 
Summary  Report  (VSR).  For  details  of  the  testing  process  see  Section  3,  Metafile  testing 
program;  Section  4,  Generator  testing  program;  and  Section  5,  Interpreter  testing  program. 

If  the  validation  procedures  are  followed  and  the  VSR  shows  that  the  lUT  demonstrated 
conformance  to  the  standard  and  the  application  profile,  then  a certificate  is  issued  to  the  client. 
Thus,  a certificate  is  only  issued  if  there  are  no  errors  detected  by  the  validation  process.  A 
Registered  Report  (i.e.,  VSR)  without  a certificate  will  be  issued  for  generators  and  interpreters 
that  have  been  tested  and  contain  errors.  Generally,  a client  must  receive  a Certificate  of 
Validation  or  Registered  Report  in  order  for  the  client’s  implementation  to  be  procured  by 
Federal  agencies. 

A certificate  received  as  a result  of  metafile  testing  is  valid  indefinitely.  A certificate  received 
as  a result  of  generator  or  interpreter  testing  is  valid  for  two  years.  A Registered  Report, 
without  a certificate,  is  valid  for  one  year. 

2.1.2  Validation  Test  Software,  Generator  and  Interpreter  Test  Suites 

MetaCheck*  with  the  MetaCALS  Option  has  been  designated  as  the  official  Validation  Test 
Software.  The  lUT  is  tested  using  the  latest  version  of  the  Validation  Test  Software,  Generator 
Test  Suite,  or  Interpreter  Test  Suite,  whichever  are  applicable.  New  releases  or  versions  of  the 
test  software  and/or  test  suites  will  be  issued  to  correct  existing  errors,  enhance  test  routines, 
and  reflect  changes  made  to  the  FIPS  or  application  profile. 

The  Generator  Test  Suite  consists  of  a questionnaire,  test  specification,  and  supporting 
documentation  that  was  developed  at  NIST.  The  Interpreter  Test  Suite  consists  of  a collection 
of  CGM  files,  operator  test  script,  and  set  of  reference  pictures.  The  client  must  sign  a site 
license  for  use  of  either  the  Generator  Test  Suite  or  the  Interpreter  Test  Suite. 


^MetaCheck  is  a licensed  product  of  CGM  Technology  Software,  P.O.Box  648,  Gales  Ferry,  CT  06335. 
MetaCheck  with  the  CALS  Option  is  referred  to  as  MetaCALS. 
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2.1.3  Renewal  of  a Certificate  of  Validation 


A Certificate  of  Validation  may  be  renewed  for  additional  years,  if  the  following  conditions  are 
met: 

1.  the  client  certifies  that  no  changes  have  been  made  to  any  component  of  the 
implementation, 

2.  the  client  certifies  that  any  changes  made  in  the  supporting  operating  system  do  not  alter 
the  function  or  operation  of  the  implementation,  and 

3.  the  Validation  Test  Software,  Generator  and/or  Interpreter  Test  Suites  have  not  changed 
substantially  since  the  last  validation  was  performed. 

Certificate  renewal  is  not  applicable  to  metafile  testing,  since  these  certificates  do  not  expire. 

2.1.4  Validation  Summary  Report  (VSR) 

The  Validation  Summary  Report  (VSR)  is  the  test  report  presenting  the  results  of  the  validation. 
The  VSR  contains  information  about  the  client,  validation  test  software  and  test  suite  versions, 
as  well  as  errors  that  may  have  been  detected.  Additionally,  the  VSR  may  contain  other 
information  gathered  during  the  validation  process,  such  as  the  profile  conformance  statement. 

A draft  VSR  and  Notification  of  Conformance  form  is  sent  to  the  client.  The  client  should 
review  the  report  and  return  the  signed  notification  form  to  NIST.  Once  the  signed  notification 
form  is  received,  the  VSR  becomes  final,  is  designated  as  a Registered  Report,  and  is  entered 
on  the  Validated  Products  List. 

For  metafile  testing,  the  finalized  VSR  is  issued  only  for  metafiles  which  are  in  compliance  with 
the  FIPS  128-1  and  MIL-D-28()03A.  References  to  all  non-compliant  CGMs  will  be  deleted 
fi-om  the  VSR  prior  to  the  VSR’s  designation  as  a Registered  Report. 

For  generator  and  interpreter  testing,  a VSR  will  be  issued  for  all  completed  validations. 

Until  the  VSR  is  finalized,  all  information  concerning  the  validation  is  considered  to  be 
confidential.  If  the  client  does  not  wish  to  release  the  VSR  information,  the  notification  form 
should  not  be  signed  or  returned  to  NIST.  No  further  action  regarding  the  validation  will  be 
taken. 
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2.2  Registration 

2.2.1  Overview 

The  rationale  for  validation  by  registration  is  that  an  implementation  may  function  identically 
on  multiple  computer  system  environments.  As  an  alternative  to  NIST  formally  validating  each 
and  every  environment,  validation  by  registration  allows  the  client  to  self-test  implementations 
on  additional  computer  system  environments.  Validation  by  registration  provides  the  client  a 
low  cost  method  for  testing  these  additional  environments  and  registering  them  in  the  VPL. 

Validation  by  registration  is  only  available  for  clients  whose  generators  or  interpreters  have  been 
formally  validated  by  NIST. 

The  process  of  validation  by  registration  consists  of  a client  conducting  the  conformance  tests, 
evaluating  the  results,  and  sending  the  results  to  NIST  for  inspection.  In  order  to  perform  the 
self-testing,  the  client  must  have  a copy  of  the  latest  version  of  the  appropriate  test  suite  and  test 
software. 

If  the  registration  criteria  are  met,  the  additional  environments  are  added  to  the  Validated 
Products  Lists  as  a registered  environment.  For  details  on  the  registration  process,  see  section 
4.4,  Registration  of  generator  environments,  and  section  5.4,  Registration  of  interpreter 
environments. 

All  self-tested  environments  are  subject  to  challenge  by  NIST  and  other  interested  parties  (such 
as  procuring  Federal  agencies).  If  the  NIST  inspection  reveals  that  a self-tested  environment 
does  not  behave  in  accordance  with  the  submitted  validation  material,  all  entries  in  the  VPL  for 
self-tested  environments  dependent  on  the  formally  validated  implementation  are  stricken. 

2.2.2  Eligibility  for  registration 

The  computer  system  environments  which  may  be  registered  include: 

1.  other  releases/versions  of  the  same  operating  system, 

2.  different  models  of  the  same  manufacturer  computer  family  (e.g.,  HP  9000  series  model 
825,  835,  850;  or  IBM  43xx,  30xx,  93xx), 

3.  different  micro-processor  architectures  from  the  same  manufacturer,  where  binary 
executable  compatibility  has  been  maintained, 

4.  re-badged  or  renamed  hardware  (i.e. , same  hardware  sold  under  a different  name).  Note, 
there  will  be  no  additional  fee  for  registering  re-badged  hardware. 
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2.3  Miscellaneous 


2.3.1  Pricing 

The  CGM  Test  Service  validations  are  to  be  performed  on  a cost-reimbursable  basis.  Pricing 
information  can  be  found  in  the  following  testing  program  sections  or  may  be  obtained  by 
contacting  the  NIST  CGM  Test  Service. 

2.3.2  Cancellation 

Once  the  validation  process  has  begun,  the  client  agrees  to  reimburse  NIST/CSL  for  the 
expenses  incurred  in  preparation  or  performance  of  the  validation.  In  the  event  that  NIST/CSL 
cancels  the  validation  due  to  nonsupport  by  the  client  or  failure  of  the  client  to  perform  in  a 
reasonable  manner,  the  client  agrees  to  pay  NIST/CSL  for  all  validation  expenses. 

2.3.3  Disputed  and  withdrawn  tests 

Questions  regarding  the  interpretation  of  the  standard  and  the  validity  of  the  tests  should  be 
forwarded  to  the  testing  laboratory  at  NIST,  along  with  associated  rationale  and  detailed 
documentation.  The  testing  laboratory  and  client  will  attempt  to  resolve  these  issues  informally. 
If  no  resolution  is  reached,  the  question  is  referred  to  the  Control  Board  for  a ruling. 

All  test  results  issued  by  the  testing  laboratory  remain  in  force  unless  and  until  reversed  by  the 
Control  Board.  If  the  test  is  judged  to  be  invalid,  the  offending  test  will  be  corrected  or 
withdrawn,  and  the  VSR  altered  to  reflect  the  ruling. 

2.3.4  Validated  Products  List 

NIST  publishes  on  a quarterly  basis,  a list  of  validated  products  having  a current  validation 
certificate  and/or  a Registered  Report.  The  Validated  Products  List  may  be  obtained  by  request 
from: 

National  Technical  Information  Service 

United  States  Department  of  Commerce 

5285  Port  Royal  Road 

Springfield,  VA  22151 

Phone:  (703)  487-4630 

Order  Number:  PB93937303/AS 

2.3.5  Documentation 

The  testing  laboratory  will  create  and  maintain  a client  file  consisting  of  the  following 
documents: 
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— all  completed  forms 

— all  correspondence,  and 

— a log  of  all  material  events  occurring  during  the  validation. 

The  client  file  and  validation  materials  will  be  retained  for  6 months  after  the  validation  process 
is  concluded.  However,  the  VSR  will  be  retained  for  a minimum  of  2 years. 

2.3.6  Publication 

In  general,  NIST  shall  have  the  right  to  use  all  information  gathered  in  the  course  of  developing 
and  administering  a conformance  testing  program  for  any  governmental  purpose.  Registered 
Reports  completed  by  NIST  shall  be  made  available  to  the  public  upon  request. 
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3 MetaHle  testing  program 


3.1  Objective 

Metafile  conformance  testing  focuses  on  testing  an  instance  of  a CGM  for  conformance  to 
Version  1 CGM  as  specified  in  the  FIPS  128-1  and  the  CALS  CGM  AP.  If  the  CGM  tested  is 
in  compliance  with  the  FIPS  128-1  and  the  CALS  CGM  AP,  a Certificate  of  Validation  will  be 
issued.  The  certificate  is  valid  indefinitely,  i.e.,  it  does  not  expire.  If  a metafile  is  modified 
in  any  way,  it  will  be  considered  a ’new’  CGM  and  thus,  not  covered  by  the  certificate. 

3.2  Testing  steps 

There  are  five  steps  that  must  be  followed  in  order  for  a metafile  to  be  validated  for 
conformance. 

1.  Request  for  validation 

To  submit  CGMs  for  testing,  the  client  completes  the  Request  for  Validation  form  and 
sends  the  request  along  with  the  CGMs  to  be  tested  and  payment  to  NIST  Computer 
Systems  Laboratory  (CSL). 

Each  CGM  submitted  to  the  test  service  shall  be  uniquely  named  and  include  the 
metafile’s  size  (in  bytes)  and  date  of  creation.  Additionally,  the  identity  of  the  CGM 
generator  as  well  as  the  hardware  platform  and  operating  system  software  must  be 
provided. 

The  CGM  to  be  tested  should  be  submitted  to  NIST  in  one  of  the  following  formats: 

PC-DOS  format  5V4"  or  2>W'  disks 

Macintosh  format  3V^''  disks 

VAX/ VMS  backup  format  16(X)  bpi,  magnetic  tape 
UNIX- tar  format  ^4 " cartridge  tape 

Other  formats  may  be  available  upon  request. 

Note  — The  CGM  should  be  represented  as  a continuous  byte  stream  using  the  binary 
encoding  as  defined  in  ANSI/ISO  8632  part  3. 

Upon  receipt  of  a testing  request  and  payment,  NIST  will  schedule  validation  and  notify 
the  client. 
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Formal  validation 


Formal  validation  takes  place  at  NIST/CSL. 

Each  CGM  is  individually  tested  using  the  latest  version  of  the  Validation  Test  Software. 
The  CGMs  are  tested  for  conformance  to  the  FIPS  128-1  and  CALS  CGM  AP. 

3.  Draft  Validation  Summary  Report 

The  test  laboratory  prepares  a draft  VSR  summarizing  the  testing  procedures  and  results 
for  each  lUT.  In  addition  to  general  information  about  the  lUT,  the  report  provides  a 
list  of  the  tests  performed  and  errors  found.  All  CGM  conformance  violations  in  the 
lUT  are  identified  and  related  back  to  the  requirements  set  forth  in  the  CGM  standard 
and  the  CALS  CGM  AP.  Finally,  the  report  clearly  indicates  whether  or  not  the  lUT 
meets  the  conformance  requirements  of  the  standard. 

4.  Notification  of  Conformance 

The  draft  VSR  along  with  a Notification  of  Conformance  is  sent  to  the  client.  The 
Notification  of  Conformance  lists  only  those  metafiles  which  are  in  compliance  with  the 
FIPS  128-1  and  the  CALS  CGM  AP.  Moreover,  these  metafiles  and  Aeir  test  results 
will  be  part  of  the  VSR  and  listed  on  the  Certificate  of  Validation. 

The  client  has  30  days  to  review  the  draft  VSR  and  notification  and  notify  NIST/CSL  in 
writing  of  any  objections.  If  there  are  no  objections,  the  client  returns  the  signed 
Notification  of  Conformance.  Receipt  of  this  notification  is  required  before  the 
Certificate  of  Validation  and  finalized  VSR  can  be  issued.  Note,  the  finalized  VSR  is 
designated  as  a Registered  Report  and  issued  as  such. 

5.  Certificate  of  Validation  and  Registered  Report 

The  Certificate  of  Validation  and  Registered  Report  (i.e.,  finalized  VSR)  will  be  issued 
for  metafiles  which  have  successfully  passed  the  conformance  requirements  of  the  FIPS 
128-1  and  CALS  CGM  AP.  The  Certificate  of  Validation  and  Registered  Report  results 
will  be  published  in  the  Validated  Products  List. 
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3.3  Pricing 

The  cost  of  testing  CGMs  is  as  follows: 


Metafile  Pricing  Table 

Number  of 
Metafiles 

PRICE  for  clients  with 
MetaCALS  license 

PRICE  for  clients  withOUT 
MetaCALS  license 

1 -5 

$500.00 

$800.00 

6-  10 

$900.00 

$1200.00 

11  - 15 

$1300.00 

$1690.00 

16  or  more 

$1800.00 

$2340.00 

If  the  submitted  metafiles  do  NOT  fit  on  a single  (1)  disk/tape,  there  is  an  additional 
charge  of  $50.00  for  each  additional  disk/tape. 
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4 Generator  testing  program 


4.1  Objective 

Generator  conformance  testing  focuses  on  testing  a CGM  generator  for  conformance  to  Version 
1 CGM  as  specified  in  the  FIPS  128-1  and  the  CALS  CGM  AP.  A Validation  Summary  Report 
(VSR)  will  be  produced  at  the  end  of  the  testing  process.  If  no  errors  are  detected  by  the  testing 
process,  a Certificate  of  Validation  and  VSR  will  be  issued.  The  Certificate  of  Validation  is 
valid  for  two  years.  If  errors  are  detected  during  the  validation,  the  Registered  Report  will  list 
the  errors  and  be  issued  as  a Registered  Report.  The  Registered  Report  is  listed  in  the  Validated 
Products  List  (VPL)  and  is  valid  for  one  year. 

Generators  are  tested  in  conjunction  with  a specific  environment.  The  environment  consists  of 
the  computer  hardware,  operating  system,  and  any  graphics  software  and  programs  required  to 
support  the  generator  under  test.  The  Certificate  of  Validation  and  Registered  Report  issued  as 
a result  of  validation  pertain  only  to  the  generator  and  environment  actually  tested. 


4.2  Testing  steps 

The  validation  process  for  generator  testing  is  a multi-step  process  conducted  in  three  phases  (see 
figure  1)  that  is  initiated  by  a client’s  Request  for  Validation. 

1.  Information  Collection 

2.  Prevalidation 

a.  Test  suite  specification 

b.  CGM  generation 

3.  Formal  Validation 

a.  On-site  validation 

b.  Test  report  summary 

c.  Notification  of  conformance 

d.  Certificate  and  VSR 

In  order  to  complete  the  testing  process,  a client  must  have  extensive  knowledge  of  its  product 
and  use  of  CGM. 
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1. 


Information  Collection 


The  client  is  provided  with  a questionnaire  regarding  the  functionality  of  the  CGM 
generator  to  be  tested.  The  completed  questionnaire  constitutes  a profile  implementation 
conformance  statement  (PICS),  stating  the  capabilities,  values,  and  options  which  have 
been  implemented  as  well  as  the  features  which  have  been  omitted. 

The  PICS  is  reviewed  for  consistency  and  violations  of  the  FIPS  128-1  or  the  MIL-D- 
28003A.  All  discrepancies  are  documented  and  resolved. 

2.  Prevalidation  Phase 

a.  Test  suite  specification  — 

Based  on  the  PICS,  a client-specific  test  specification  is  defined.  The  test 
specification  consists  of  test  scripts  and  test  images. 
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Figure  1.  Testing  process 
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b.  CGM  generation  — 

Using  the  test  specifications,  the  client  produces  the  requisite  computer  graphics 
metafiles.  Some  lUTs  may  require  modification  to  the  test  specifications.  The 
client  must  submit  any  proposed  modifications  to  NIST  for  approval.  All 
approved  modifications  shall  be  documented. 

The  client  submits  the  CGMs  and  supporting  documentation  to  NIST  for 
prevalidation  inspection.  Upon  completion  of  prevalidation,  the  schedule  and 
arrangements  for  on-site  validation  are  confirmed. 

3.  Formal  Validation 

a.  On-site  validation  — 

The  on-site  validation  consists  of  NIST  personnel  witnessing  the  creation  of  the 
CGMs  from  the  native  picture  files  or  programs. 

The  CGMs  are  taken  back  to  NIST,  where  they  are  checked  for  completeness  to 
ensure  that  all  the  test  suite  requirements  have  been  met  and  syntactic  and 
semantic  correctness  have  been  achieved.  The  syntax  is  tested  using  the  latest 
version  of  the  Validation  Test  Software,  MetaCALS. 

b.  Test  Report  Summary  — 

NIST/CSL  prepares  a draft  VSR  summarizing  the  testing  procedures  and  results 
of  the  generator  testing.  The  report  includes  information  about  the  generator  and 
the  environment  under  which  it  was  tested,  the  PICS,  and  all  errors  (behavior  not 
in  conformance  with  the  standard,  application  profile,  or  PICS).  Finally,  the 
report  clearly  indicates  whether  or  not  the  lUT  meets  the  conformance 
requirements  of  the  FIPS  128-1  and  the  CALS  CGM  AP  and  the  functionality  as 
claimed  by  the  vendor  in  the  PICS. 

c.  Notification  of  Conformance  ~ 

The  draft  VSR  along  with  a Notification  of  Conformance  is  sent  to  the  client. 
The  Notification  of  Conformance  lists  the  generator  and  environment  for  which 
the  certificate  and  final  VSR  will  be  issued.  Additionally,  any  errors  and/or 
conditionally  supported  capabilities  will  be  identified. 

The  client  has  30  days  to  review  the  draft  VSR  and  notification  and  notify 
NIST/CSL  in  writing  of  any  objections.  If  there  are  no  objections,  the  client 
returns  the  signed  Notification  of  Conformance.  Receipt  of  this  notification  is 
required  before  the  Certificate  of  Validation  and  final  VSR  can  be  issued.  Note, 
the  finalized  VSR  is  designated  as  a Registered  Report  and  issued  as  such. 
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d.  Certificate  of  Validation  and  Registered  Report  ™ 

A Certificate  of  Validation  will  be  issued  for  generators  which  have  successfully 
passed  the  conformance  requirements  of  both  the  FIPS  128-1  and  CALS  CGM 
AP,  and  meet  the  functionality  as  claimed  in  the  PICS.  A Registered  Report, 
without  a certificate,  will  be  issued  for  generators  which  have  been  tested  and 
contain  errors.  The  certificate  and  Registered  Report  will  be  published  in  the 
Validated  Products  List. 


4.3  Pricing 

The  cost  for  CGM  generator  testing  is  as  follows: 


Generator  Pricing  Table 

Validation 

Registration 

$9,000  — base  price 

$250  — for  each  environment  (1  to  7) 

$600  — for  each  additional 
environment 

$2,000  —■  for  8 or  more 
environments 

$6,000  — for  each  additional 
generator 

4.4  Registration  of  generator  environments 

Clients  whose  generators  have  been  formally  validated  by  NIST  can  register  (i.e.,  list  in  the 
VPL)  the  generator  on  other  computer  system  environments  based  on  the  client’s  self-testing. 

Registration  of  generator  environments  is  applicable  only  if  the  generator,  when  implemented 
on  other  computer  system  environments,  is  the  same  executable  code  and  uses  the  same 
application  software  or  calling  program  to  create  input  to  the  generator  as  that  used  for  the 
validation  by  testing  process. 

To  register  a generator  for  processing  on  other  computer  system  environments,  the  client  must 
submit  a request  for  registration.  The  request  must  be  made  before  the  generator’s  validation 
expires. 

The  client  performs  the  prevalidation  step  described  above.  Prior  to  submitting  the  CGMs  to 
NIST,  the  client  checks  that  the  files  are  semantically  the  same  as  the  original  lUT’s  CGMs  and 
using  the  Validation  Test  Software,  checks  the  syntax  of  the  files.  The  client  submits  a signed 
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statement  along  with  the  validation  results  (i.e.,  CGMs)  affirming  that  the  CGMs  are  the  result 
of  executing  the  tests  in  the  specified  environment. 

NIST  inspects  the  validation  results.  If  the  generator,  when  implemented  on  the  additional 
environment,  is  functionally  equivalent  to  the  generator  implemented  on  the  environment  used 
during  the  validation  by  testing  process,  then  the  environment  will  be  registered  in  the  VPL. 
Additionally,  the  client’s  signed  statement  regarding  the  registered  environment  is  added  to  the 
VSR. 
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5 Interpreter  testing 


5.1  Objective 

Interpreter  conformance  testing  focuses  on  testing  a CGM  interpreter  for  conformance  to  Version 
1 CGM  as  specified  in  the  FIPS  128-1  and  the  CALS  CGM  AP.  A Validation  Summary  Report 
(VSR)  will  be  produced  at  the  end  of  the  testing  process.  If  no  errors  are  detected  by  the  testing 
process,  a Certificate  of  Validation  and  VSR  will  be  issued.  The  Certificate  of  Validation  is 
valid  for  two  years.  If  errors  are  detected  during  the  validation,  the  VSR  will  list  the  errors  and 
be  issued  as  a Registered  Report.  The  Registered  Report  is  listed  in  the  VPL  and  is  valid  for 
one  year. 

Interpreters  are  tested  in  conjunction  with  a specific  environment.  The  environment  consists  of 
the  computer  hardware,  output  devices  (display  or  printer),  operating  system,  and  any 
application  software  or  programs  required  to  support  the  implementation  under  test  (lUT).  The 
certificate  and  Registered  Report  issued  as  a result  of  validation  pertain  only  to  the  interpreter 
and  environment  actually  tested. 


5.2  Testing  steps 

The  validation  process  for  interpreter  testing  is  a multi-step  process  conducted  in  two  phases  that 
is  initiated  by  a client’s  Request  for  Validation. 

1.  Prevalidation 

a.  Test  Suite  installation  and  interpretation 

b.  Completion  of  Operator  Test  Script 

2.  Formal  Validation 

a.  On-site  validation 

b.  Test  report  summary 

c.  Notification  of  conformance 

d.  Certificate  and  VSR 


1.  Prevalidation  Phase 

a.  Test  Suite  installation  and  interpretation  — 

The  client  is  provided  with  the  Interpreter  Validation  Test  (TVaT)  suite.  The 
rVaT  suite  consists  of  a collection  of  CGM  files,  an  operator  script,  and  a set  of 
reference  pictures.  The  metafiles  are  copied  from  the  (NIST  supplied)  disk  to  the 
client’s  system.  Each  metafile  must  be  interpreted  using  the  lUT.  The  resulting 
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picture  is  then  compared  to  the  reference  picture  and  a pass/fail  decision  is  made 
based  on  the  criteria  described  in  the  operator  script. 

b.  Completion  of  Operator  Test  Script  -- 

The  client  submits  the  completed  operator  test  script  for  prevalidation  inspection. 
Upon  completion  of  prevalidation,  the  schedule  and  arrangements  for  on-site 
validation  are  confirmed. 

Formal  Validation  Phase 

a.  On-site  validation  — 

The  on-site  validation  consists  of  the  NIST  personnel  witnessing  the  installation 
and  execution  of  a ’new’  copy  of  the  IVaT  suite. 

b.  Draft  VSR  - 

NIST/CSL  prepares  a draft  VSR  summarizing  the  testing  procedures  and  results 
of  the  interpreter  testing.  The  report  includes  information  about  the  interpreter 
and  the  environment  under  which  it  was  tested,  and  all  errors  (behavior  not  in 
conformance  with  the  standard  and  application  profile).  Finally,  the  report 
clearly  indicates  whether  or  not  the  interpreter  meets  the  conformance 
requirements  of  FIPS  128-1  and  the  CALS  CGM  AP. 

c.  Notification  of  Conformance  ~ 

The  draft  VSR  along  with  a Notification  of  Conformance  is  sent  to  the  client. 
The  Notification  of  Conformance  lists  the  interpreter  and  environment  for  which 
the  Certificate  and  final  VSR  will  be  issued.  Additionally,  any  errors  and/or 
conditionally  supported  capabilities  will  be  identified. 

The  client  has  30  days  to  review  the  draft  VSR  and  notification  and  to  notify 
NIST/CSL  in  writing  of  any  objections.  If  there  are  no  objections,  the  client 
returns  the  signed  Notification  of  Conformance.  Receipt  of  this  notification  is 
required  before  the  Certificate  of  Validation  and  final  VSR  can  be  issued.  Note, 
the  finalized  VSR  is  designated  as  a Registered  Report  and  issued  as  such. 

d.  Certificate  of  Validation  and  Registered  Report  — 

A Certificate  of  Validation  will  be  issued  for  interpreters  which  have  successfully 
passed  the  conformance  requirements  of  both  the  FIPS  128-1  and  CALS  CGM 
AP.  A Registered  Report,  without  a certificate,  will  be  issued  for  interpreters 
which  have  been  tested  and  contain  errors.  The  Certificate  of  Validation  and 
Registered  Report  will  be  published  in  the  Validated  Products  List. 


5.3  Pricing 

The  cost  for  CGM  interpreter  testing  is  as  follows: 


Interpreter  Pricing  Table 

Validation 

Registration 

$5,000  — base  price 

$250  — for  each  environment  (1  to  7) 

$1,500  — for  each  additional 
environment 

$2,000  — for  8 or  more 
environments 

$1,500  — for  each  additional 
interpreter 

5.4  Registration  of  interpreter  environments 

Clients  whose  interpreters  have  been  formally  validated  by  NIST  can  register  (i.e.,  list  in  the 
VPL)  the  interpreter  on  other  computer  system  environments  based  on  the  client’s  self-testing. 

Registration  of  interpreter  environments  is  applicable  only  if  the  interpreter,  when  implemented 
on  other  computer  system  environments,  is  the  same  executable  code  and  uses  the  same 
application  software  or  calling  programs  to  interpret  the  metafiles  as  that  used  for  validation  by 
the  testing  process. 

To  register  an  interpreter  for  processing  on  other  computer  system  environments,  the  client  must 
submit  a request  for  registration.  The  request  must  be  made  before  the  interpreter’s  validation 
expires. 

i 

The  client  performs  the  prevalidation  step  described  above.  The  client  shall  interpret  each 
metafile,  compare  the  resulting  translation  to  the  reference  picture,  and  complete  the  operator 
script.  The  client  submits  a signed  statement  along  with  the  operator  script  affirming  tiiat  the 
validation  results  were  obtained  by  executing  the  tests  in  the  specified  environment. 

NIST  inspects  the  validation  results  contained  in  the  operator  script.  If  the  interpreter,  when 
implemented  on  the  additional  environment,  is  functionally  equivalent  to  the  interpreter 
implemented  on  the  environment  used  during  the  validation  by  testing  process,  then  the 
environment  will  be  registered  in  the  VPL. 
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